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1 Introduction 


National Treasury is driving a local government financial and budgeting reform 
process. This intends 

• To modernise local government budgeting and financial management 
processes and practices 

• To improve financial governance by clarifying and separating roles and 
responsibilities of Mayors, executive and non-executive councillors vis-a-vis 
those of municipal officials. 

• To maximise municipal capacity to deliver services by attending to issues of 
efficiency, effectiveness and sustainability, and dealing with corruption. 

• To set-up the accountability cycle by ensuring proper linkages between 
IDPs, budgets, SDBIPs, in-year reports, annual financial statements, annual 
reports, oversight reports and audit reports. 

Specifically, amongst other objectives it will also seek to 

• Strengthen the links between transparency, understandability and 
accountability 

• Promote 'good practice' in municipal financial reporting through: 

promoting simplicity ahead of complexity 

- ensuring standardisation across the local government sphere 

- developing a standardised vote structure to be approved by municipal 
councils 

minimising the cost of compliance and information gathering 

- ensuring consistency and comparability with other spheres of 
government 

• Ensure information is available to: 

- determine the financial status of municipalities 

- assess if municipal budgets are funded 

- assess indigent policies and pro-poor equity issues 

- facilitate performance comparisons and evaluations 

• Promote improved serviced delivery 


The City Support Programme of National Treasury has initiated a project on 
reporting reforms for Cities, particularly in relation to built environment functions. 

This constitutes an emerging framework for reporting on built environment 
functions of cities. It follows the first phase of work, which assessed the current 
status of city reporting, primarily in relation to built environment reporting. 

This document provides a 

• A conceptual framework for built environment reporting that will need to be 
to be completed, or progressively realised, over time 
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• A set of design principles for intergovernmental M&E systems to align to, 
individually and collectively 

• A proposed set of criteria to ensure that we have quality indicators. All 
supported indicators will need to comply with all of these criteria. 


These principles, criteria and conceptual framework are now complemented by a 
proposed systemic framework, institutional arrangements and responsibilities to 
complete a framework for built environment reporting for metropolitan 
municipalities. 


2 Background 


2.1 Constitutional mandate 

Local government derives its mandate from Chapter 7 of the Constitution which 
sets out its establishment as the third sphere of government with its own powers 
and functions. As a sphere of government, local government is expected to function 
cooperatively with national and provincial government. Chapter 3 of the 
Constitution sets out the principles of cooperative governance that enable the three 
spheres of government to co-exist as "distinctive, interdependent and inter-related 
entities. 

In addition, section 154 (1) of the Constitution provides for national and provincial 
government to support and strengthen the capacity of municipalities to manage 
their own affairs, exercise the powers and perform their functions. To this end, the 
local government sector is constituted by the Department of Cooperative 
Governance; its 9 provincial chapters; and municipalities. 

The constitution is written in the spirit of subsidiarity, embracing decentralisation 
and the idea that governance should take place as close as possible to the citizens. 
This is demonstrated in two ways. Firstly, it is made clear in the Constitution that 
national and provincial government may not interfere in local government without 
good reason. Secondly Section 156(4) provided that national or provincial 
government "must" assign certain functions to local government when certain 
specified conditions are met^ 

In accepting the concept of subsidiarity there is the recognition that the state is not 
the same in all places and that asymmetry or differentiation is warranted. 
Furthermore devolution in a developmental state means vigilant monitoring for 
malfunction and correction where necessary. In order to make rational 
adjustments, however there is need for an informed basis from which to act. 
Therefore, there is a need for good information on the state of local government 
broadly, and each municipality specifically, which needs to be generated through 
monitoring. The principle of subsidiarity and decentralisation makes necessary the 
need for good information and hence good monitoring practise. 

2.2 The White Paper on Local Government, 1998 

The White Paper on Local Government builds on the context for monitoring, support 
and intervention as set out in the Constitution and further describes the roles and 


^ Community Law Centre (2008) Institutional Subsidiarity in the Constitution: Slapstick 
asymmetry or "rights-based" approach to powers? 
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responsibilities of national and provincial government with respect to local 
government. 

National Government's role 

National Government's role includes providing an overall framework for municipal 
capacity-building and support as well as an overall framework for a system of 
monitoring and oversight within which other organs of state, particularly provincial 
governments will perform these functions. 

The White Paper notes that whilst intervention into municipalities is primarily a 
provincial government responsibility, national government may need to intervene 
together with provincial government, or when provincial government fails to 
intervene. 

2.3 Local Government Legislation 

The Municipal Systems Act and the Municipal Finance Management Act further build 
on the policy imperative as enunciated in the White Paper on Local Government. 
These are further explained below: 

2.3.1 Municipal Systems Act, 2000 

The Municipal Systems Act has, been the key instrument in introducing both 
performance management and monitoring and evaluation practices with respect to 
local government. 

The basis of reporting requirements for the Co-operative Governance are found in 
the in the Municipal Systems Act 32 of 2000. Section 38 of this Act requires that 
each municipality develop a performance management system that: 

• Set targets, monitor and review performance based on indicators linked to 
their IDP 

• Publish an annual report on performance for the councillors, staff, the public 
and other spheres of government 

• Incorporate and report on a set of general indicators prescribed nationally by 
the minister responsible for local government 

• Conduct an internal audit on performance before tabling the report. 

• Have their annual performance report audited by the Auditor-General 

• Involve the community in setting indicators and targets and reviewing 
municipal performance^ 

Additionally a municipality must establish a mechanism to monitor and review its 
performance management system. 

Sections 46, 47 and 48 of the Municipal Act set out the requirements for 
performance reporting. 

Section 46 of the Municipal Systems stipulates that a municipality must prepare a 
performance report each financial year that reflects: 

a. the performance of the municipality and of each external service 
provider during that financial year; 


^ Local Government Municipal Systems Act 32 of 2000 section 38 
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b. a connparison of the perfornnances referred to in paragraph (a) with 
targets set for and perfornnances in the previous financial year; and 

c. measures taken to improve performance. 

This annual performance report must form part of the municipality's annual report 
in terms of Chapter 12 of the Municipal Finance Management Act.^ 

The MSA, in section 47, requires the MEC for local government of a province to 
annually compile and submit to the provincial legislatures and the Minister a 
consolidated report on the performance of municipalities in the province. The report 
must: 


a. identify municipalities that under-performed during the year; 

b. propose remedial action to be taken; and 

c. be published in the Provincial Gazette. 

A copy of this report must also be submitted to the National Council of Provinces.'^ 

Lastly, section 48 of the Municipal Systems Act requires that the Minister for Local 
Government must annually compile and submit to Parliament and the MECs for 
local government a consolidated report of local government performance in terms 
of general key performance indicators. This report must be published in the 
Gazette.^ 

Performance management as laid out in the Municipal Systems Act creates an 
inherent challenge for monitoring local government performance. The system 
decentralising goal, indicator setting and monitoring in section 38, and require each 
municipality to establish its own indicators based on its IDP developed with 
extensive public participation. However, Section 47 reports by the MEC require a 
consolidated performance report of all its municipalities, each of which will have its 
own individual set of indicators. The problem is further compound for the Section 
48 report by the Minister, where the legislation demands a consolidated report 
based on general key performance indicators prescribed in regulations. 

2.3.2 Municipal Finance Management Act, 2003 

The Municipal Finance Management Act 56 of 2003 (MFMA), Chapter 13, Section 71 
of the Municipal Financial Management Act ( initiated December 2004) designates 
the accounting officer of the municipality to submit monthly budget statements to 
Provincial Treasury, no later than 10 working days after the end of the month. The 
budget statements must state the actuals for that month, as well as the financial 
year to date. The budgetary information included is: 

• Actual revenue per revenue source 

• Actual borrowings 

• Actual expenditure per vote 

• Actual capital expenditure per vote 

• Amount of any allocations received 

• Actual expenditure of those allocations 


^ Local Government Municipal Systems Act 32 of 2000 section 46 

^ Local Government Municipal Systems Act 32 of 2000 section 47 

^ Local Government Municipal Systems Act 32 of 2000 section 48 
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• Where necessary, an explanation of any variations from the projected 
revenue or expenditure 

Provincial Treasury has then 22 working days after the end of the month to submit 
a consolidated statement to National Treasury for all the municipalities and 
municipal entities in their jurisdiction. Within 30 days of the end of each quarter, a 
consolidated statement must be released by the Provincial Treasury on the state of 
the municipalities and municipal entities' finances. 

2.4 Monitoring Reporting and Evaiuation Poiicy Context 

2.4.1 Government-Wide Monitoring and Evaluation 

The 2007 Policy Framework for the Government-Wide Monitoring and Evaluation 
System^ (GWM&E) set out the details of the GWM&E system, at an institutional 
level along with the roles and responsibilities for implementing the system. 

While under review, the document remains the overarching policy framework for 
M&E in the South African government and provides the context for the supporting 
frameworks that act as the three pillars of GWM&E, the National Treasury's 
Framework for Managing Programme Performance Information and StatsSA's South 
African Statistics Quality Assurance Framework and DPME's National Evaluations 
Policy Framework, all reflected upon in subsequent sections. 

The GWM&E is applicable to all entities across the spheres of government. 

The framework outlines an M&E system as a set of organisational structures, 
management processes, standards, strategies and plans, indicators, information 
systems, reporting lines and accountability relationships that allow government 
institutions to discharge their M&E functions effectively. It specifies that the 
framework is not about IT systems but is a policy context in which IT systems will 
operate. 

2.4.2 Framework for Managing Programme Performance Information 

The Framework for Managing Programme Performance Information (FMPPI) 
released in 2007 makes National Treasury the primary responsible institution for 
performance information collected from government institutions in the process of 
fulfilling their mandates and implementing policies. This information includes details 
of outputs and outcomes. The aims of the FMPPI are to clarify standards for 
performance information and supporting non-financial audits; improve the 
structures systems and processes for managing performance information; define 
roles and responsibilities for performance information; and promote accountability 
through timely publication of performance information. ^ 

The National Treasury is required to develop standards to implement the FMPPI; to 
develop formats for accountability reporting; to develop core sets of performance 
information in collaboration with sector departments to ensure uniformity of 


® The Presidency, 2007. Policy Framework for the Government-wide Monitoring and 
Evaluation System. The Presidency, Pretoria. 

^ National Treasury, 2007. Framework for Managing Programme Performance Information. 
National Treasury, Pretoria. 
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infornnation; and to develop guidelines on the use of performance information. In 
addition, the National Treasury, along with provincial treasuries, are required 
monitor the implementation of FMPPI by all institutions within their spheres; to 
provide training on the use of performance information; to provide inputs on the 
process for the selection of indicators; and to use the information generated by 
institution to report on the economy, efficiency, effectiveness, and equity in the use 
of resources. 

Under the FMPPI national departments responsible for concurrent functions need to 
be directly involved in the development of systems and structures to collect 
performance information on these functions across the spheres. They need to play 
a role in supporting provincial departments in managing performance information 
as well as monitor performance information themselves and evaluate delivery of 
service in their sectors. 

2.4.3 South African Statistical Quality Assessment Framework 

The purpose of the South African Statistical Quality Assessment Framework 
(SASQAF) is to provide a structure for the assessment of statistical products for 
self-assessment, reviews by the Statistics South Africa, assessment by data users 
and assessment by international agencies.® 

The SASQAF outlines the details by which statistics should be judged as being of 
good quality or not. In terms of the protocol for the designation of statistics the 
Statistician General will do so only if the statistics meet the SASQAF criteria for 
quality. The protocol also specifies that only official statistics will be used to inform 
the GWM&E system. The framework outlines 8 dimensions of quality: relevance, 
accuracy, timeliness, accessibility, interpretability, coherence, methodological 
soundness and integrity. 


3 Current Planning and Reporting Instruments 


A range of legislation has been passed governing strategic planning, monitoring and 
reporting for local government. How different planning documents relate to each 
other, and in particular their reporting formats and requirements, is critical for 
moving towards the operationalisation of all local government indicators within a 
results-based framework. The following sets out some of the basic reporting 
requirements in terms of key legislation, policy and guidelines that inform the 
different reporting and format requirements. 

3.1 Integrated Development Plan 

An Integrated Development Plan (IDP) represents the foundation of municipal 
planning as enshrined in the Municipal Systems Act (MSA) 32 of 2000 (as 
amended). The IDP makes clear the expectation that IDPs place clear 
responsibilities on the municipality to develop IDPs that include, amongst others: 
"an assessment of the existing level of development in the municipality, which must 
include identification of communities which do not have access to basic municipal 
services... .the council's development priorities and objectives for its elected 


® Statistics South Africa, 2008. The South African Statistical Quality Assessment Framework 
First Edition. Statistics South Africa, Pretoria. 


6 


Reporting Reforms Project: Draft Framework - Principles and Criteria 


ternn...the key perfornnance indicators and performance targets determined in terms 
of section 41" (RSA, 2000). 

Section 41 of the MSA stipulates that a municipality must: "Set appropriate 
performance indicators as yardstick for measuring performance, including for 
outcomes and impact, with regards to the municipality's development priorities and 
objectives set out in its IDP; 

• Set measurable performance targets for each of those priorities and 
objectives; 

• For each target, monitor and review performance at least once a year; 

• Take steps to improve performance where targets are not met; and 

• Establish regular reporting internally and to the public" (RSA, 2000). 

There is a clear relationship between the IDP and the Performance Management 
System of a municipality, and MSA Section 38 requires that a municipality must 
"establish a performance management system that is-.... (iii) in line with the 
priorities objectives, and targets contained in its IDP" (RSA, 2000). This means that 
performance indicators and targets set in the IDP need to be consistent and 
integrated with the objectives and priorities set in the IDP. 

Further, any performance management system applied by the municipality "must 
be devised in such a way that it may serve as an early warning indicator of under- 
performance," both for individuals and the organisation as a whole. 

The Local Government: Municipal Planning and Performance Management 
Regulations (RSA, 2001) set additional requirements for indicators, including that 
they should be "measurable, relevant, objective and precise" and that reporting to 
the municipal council occur at least twice a year, and include 12 prescribed 
indicators. Despite non-compliance with these regulations being regularly identified 
in Auditor-General findings as an area for improvement in many municipalities, 
these regulations appear to be increasingly disregarded. 

Thus, there is a clear requirement for IDPs that indicators are set at "outcome and 
impact" level and should be reported on an annual basis. This is in line with reviews 
and updates to the IDP and that performance tracking should allow for "an early 
warning indicator of under-performance". 

3.2 SDBIP 

The Service Delivery Budget Implementation Plan (SDBIP) was introduced and 
explained with the Municipal Finance Management Act (MFMA) Circular 13 of 2005 
(NT, 2005). In it, the circular prescribes five components of the SDBIP, which 
include the following: 

Item 1 Monthly projections of revenue to be collected for each source 

Item 2 Monthly projections of expenditure (operating and capital) and revenue 
for each vote 

Item 3 Quarterly projections of service delivery targets and performance 
indicators for each vote 

Item 4 Ward information for expenditure and service delivery 
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Itenn 5 Detailed capital expenditure plan broken down by ward over three years 

Only item 3 is the focus as it is the only element of the SDBIP related to the non- 
financial performance information. MFMA reporting requirements and the SDBIP 
Guidelines (NT, 2010) confirm that the performance information aspect of in-year 
reporting supports requirements "to provide an explanation of PI variances on a 
quarterly basis". 

Whereas there are monthly reporting requirements associated with the other items, 
performance information only need be reported on a quarterly basis, but this poses 
distinct challenges. For instance, compared to financial information, which is 
available within 10 days of month end, a lag of 2-3 months may ensue and it can 
be a challenge to determine "the appropriate and objective performance 
information for service delivery, and to measure the quality of service delivery. This 
is an art that will require managers to be more creative and innovative" (NT, 2010). 

The MFMA SDBIP Guidelines (NT, 2010) also indicates that the SDBIP needs to 
provide for the monitoring of "inputs, outputs and outcomes for each senior 
manager (department) by vote and Standard Classification (GFS) sub- 
classification". This creates the basis for linking individual performance with 
organisational performance, as well as tying it to budgeting. 

Thus, in summary, the SDBIP (as it pertains to reporting formats and 
requirements) should logically follow from the performance objectives and targets 
set in the IDP, but with a short-term focus and the requirements of quarterly 
performance projections and reporting. This is expected to cover inputs, outputs 
and outcomes. However, in reality there are few examples of municipal-wide citizen 
outcomes which a municipality could source, collect, collate and report for on a 
quarterly basis, credibly, especially that of a metropolitan municipality. The reality 
is that SDBIPs better lend themselves to municipal service output measures. 

3.3 Built Environment Performance Plan 

The Built Environment Performance Plan (BEPP) was introduced for metropolitan 
municipalities as a condition stipulated in the Division of Revenue Act No. 6 of 2011 
where its development was included as a pre-requisite to receiving the Urban 
Settlements Development Grant (USDG). The USDG alluded to a 'built environment 
performance framework' initially and later a BEPP (in RSA, 2012) which should be 
"consistent with the IDP and SDBIP of metropolitan municipalities". Later, the BEPP 
became an eligibility requirement for the Integrated City Development Grant 
(ICDG) in 2014/2015 (NT, 2013). 

The City Support Programme (CSP) BEPP Guidance Note (NT, 2013) indicates that 
the BEPP is placed in between the Municipal Spatial Development Framework 
(MSDF) and IDP, "with an explicit focus on the social and economic infrastructure 
components of the built environment as it manifests in space". 

Despite multiple performance frameworks associated with the various draft 
iterations of the USDG policy framework, there was never a legally prescribed set of 
built environment performance indicators set for BEPPs. Reporting formats set out 
by the Department of Human Settlements changed over time, were inconsistently 
applied and comprised indicators mostly at the output level. 

The CSP BEPP Guidance Note makes reference to MFMA and grant reporting 
requirements, reiterating that reporting built environment performance should be 
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integrated with existing reporting requirements and formats rather than impose 
additional reporting burden. 

As part of the shift to linking the BEPP to the ICDG, a set of built environment 
indicators were to be developed by metros by March 2014 and included in section D 
of the BEPP. The indicators were intended to be incorporated into SDBIPs, with the 
2014/15 financial year serving as the pilot year for incorporating the indicators into 
SDBIPs and integrating them within the existing planning framework (NT, 2013: 
10 ). 


The CSP BEPP Guidance note states that "Once incorporated into the SDBIPS, 
reporting on the performance of metro built environments becomes part of the 
general reporting of the metro in terms of the established regulations of National 
Treasury alongside section 71 Reports (due 10 days after the end of each month) 
and quarterly reporting against SDBIPS" (NT, 2013: 13). 

The legislative and policy frameworks informing the built environment planning, 
monitoring and reporting requirements and formats should be understood in 
relation to municipal planning documents the IDP, SDBIP and BEPP. IDPs set out 
strategic municipal priorities and objectives that should be measured on an annual 
basis (MSA) or half-yearly (MSA Planning & Performance Regulations). In terms of 
the MFMA, indicators and targets set in the SDBIP are the subject of quarterly 
reporting of non-financial performance. The BEPP introduced a strategic spatial 
planning element that is meant to be situated between the MSDF and the IDP, but 
that does not impose significant new reporting burdens as it manifests in terms of a 
set of specified indicators integrated with the SDBIP. 

The holistic picture is presented in the local government accountability cycle below 



Figure 1: The local government accountability cycle (NT) 

The planning instruments referred to above are in need of review and alignment. A 
Planning alignment task team has been established. There are significant 
implications that arise from this conceptual framework, principles and criteria that 
require review of the structure and purpose of SDBIPs and IDP's in particular. 

3.4 Auditability 

There have been significant concerns on the part of municipalities regarding the 
introduction of outcomes measures, particularly in so far that the Auditor-General is 
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able to audit this data. It has been so far agreed that this reform process will 
ensure that we work collaboratively with the office of the AG in ensuring that 
auditing approaches suits the types and purpose of information used for outcomes 
reporting. 


4 Problem Statement 


4.1 Rationalisation of LG Data Collection Processes 

In November 2007, National Treasury published a document on the Rationalisation 
of the Local Government Data Collection Processes. The research undertaken 
identified weak co-ordination and duplication in reporting, lack of alignment in 
reporting periods, inefficient technology being used for reporting, a burden of 
reporting (more than 80 questionnaires annually), limited capacity, skills and 
insight into how reporting is used. 

4.2 DPME Study into State and Use of M&E 

The Department of Planning, Monitoring and Evaluation (DPME) completed 
Research into the State and Use of M&E in Five Sectors. Local Government was 
treated as a sector, but so too were the Human Settlements and Water Services. 
The study of these three sectors provided insight into intergovernmental reporting 
done by local government. This research highlighted the following conclusions: 

Appreciate complexity 

There have been significant developments in M&E systems in the 20 years of 
democratic government in South Africa. During this time there has been much 
experience in M&E initiatives at all levels of the state. Our maturing 
intergovernmental system is far from where it needs to be and the complexity of 
implementing M&E in this developmental context needs to be fully appreciated. 

Gap between systems and legal mandate 

There is a significant gap between the M&E systems that exist and the legal 
mandates of departments, particularly with regard to their regulatory and 
supervisory roles. The systems to enable departments fulfil their core roles, rarely 
exist in more than a rudimentary form. However there are elements of initiatives 
and innovations, some of which are arguably proving successful. This holds 
particularly true for the local government, human settlements and water sectors. 

Diverse types of systems 

Government-wide approaches to M&E need to fully appreciate the different 
typologies of systems designed to meet unique sets of purposes. While supervisory 
accountability and regulation is arguably the main driver, different types of systems 
have been brought in to respond to other purposes. Other drivers for M&E include 
learning, support and improvement and citizen-based accountability. Systems have 
emerged in design to emphasise particular needs in M&E and have consequently 
realised different monitoring and reporting behaviours. For example, peer owned 
benchmarking systems, as well as vulnerability self-assessments, as found in the 
water sector, have resulted in vast voluntary information sharing as a result of a 
non-punitive intentions of these systems. 
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Need for coherent frameworks 

Particularly the gap between legal mandates and M&E functionality, as well as the 
diversity of systems that exist in each department have emphasised the need for 
coherent frameworks for M&E. 

While there are some departments with a department-wide M&E framework, these 
have been shown to be out of touch with the reality of systems that exist and the 
full mandate of the department. Where the work of the department cuts across all 
three spheres of government and includes a multitude of other agencies, such a 
sectoral framework for M&E becomes paramount. 

These intergovernmental M&E frameworks need to give expression to the 
implications of the legal mandate of the department and other institutions operating 
in the sector, identifying the array of possibly different systems that facilitate the 
full set of M&E needs in the sector coherently and clearly identify the roles and 
responsibilities of the different institutions that need to make this happen. 

Approach to M&E capacity 

Previous approaches to M&E capacity, indicate that there is an emphasis on 
dedicated M&E capacity located within a centralised unit of a department. The 
research has shown that this dedicated and centralised M&E capacity has focused 
significantly on internal M&E and under-emphasising the sector. In departments 
that fulfil a role in monitoring, supporting, regulating other entities, 
intergovernmentally, the wealth of M&E resides in line functions and in 
programmes. Decentralised capacity in programmes, however, often relies on a 
small part of a senior official's time, or a larger part of a more junior official's time. 

Universally, M&E capacity is in short supply, particularly when considering the right 
skills. 

Data governance is poor 

While there have been advances in M&E systems in government over the past two 
decades, data governance remains weak. Data governance is used here as a 
combined set of ideas. 

The first set of ideas is about how data is, not just managed, but governed, 
requiring leadership in how data is planned for, used in planning, monitoring, 
reporting and evaluation processes with integrity. It is also concerned with how 
data is quality assured, co-ordinated, shared and made accessible, maximising its 
uses and making data collection efficient. 

The research has shown that indicators are poorly specified, data needs are not well 
planned for and co-ordinated and there are huge inefficiencies and duplication in 
fragmented forms of data collection, particularly from entities like households and 
municipalities. 

The second set of ideas is about how governance takes places. The term data 
governance emphasised the need to extend evidence-based policy-making to the 
ongoing activities of management and governance. How we use data or evidence in 
all the acts of governance requires attention. 
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It is argued that underlying poor data governance is a capacity and leadership 
problem. There is an insufficient appreciation of the hard skills required to 
efficiently respond to the data needs of government, managing the quality of 
information and its usefulness and uses in all the processes of governance. 

This requires a culture shift, which requires the necessary leadership to champion 
good data governance in departments and sectors. 

Co-ordination is weak 

Related to the problem of poor data governance, is that the co-ordination of M&E 
systems and data needs is weak, across departments. 

There is significant duplication and fragmentation in data being collected, where 
entities such as municipalities argue that they experience a burden of reporting that 
is uncoordinated and duplicative. The efficiencies and effectiveness that would be 
achieved by shared and co-ordinated approaches to M&E across departments are 
convincing and in need of better co-ordination. This co-ordination is not being 
achieved across departments currently. 

A case in point relates to the much-needed data on access to services, particularly 
at household and community level. The development of an adequate results-based 
monitoring system for monitoring access to basic services infrastructure to 
households does not exist because the appropriate solution requires co-ordination 
across a number of departments. This means that in between the ten year census, 
there exists no commonly accepted system of information on which to monitor, 
manage and plan the results of R75bn of annual public expenditure on municipal 
infrastructure. 

Co-operative Ethos & Feedback 

Our constitution provides for three interrelated, interdependent yet distinct spheres 
of government, who are compelled to co-operate in governance. It is critical that in 
intergovernmental M&E there is respect for different spheres and that feedback 
obligations, whether explicit or implicit, are honoured. 

Far too often, reporting entities complain that they are compelled to report in an 
ad-hoc urgent way, where a systematic approach could have been put in place pre- 
emptively, and they 'never' receive feedback regarding what they have reported, 
raising questions about whether their reports are read or used in any way. 

It is important that in M&E systems that reporting entities are incentivised and 
rewarded with analytical, interpretive or comparative feedback that they will find 
useful and the reporting process is in itself also useful for them. 

Co-operation is not only vertical but horizontal, requiring co-operation amongst 
departments in the same sphere. 

4.2.1 Reporting Reforms Project 

The CSP has, in the project charter for this project, argued that there are two main 
reasons for this project 

• There are too many uncoordinated indicators and requests for data by 
national departments to metros. There is a need to review and rationalize 
these. 


12 


Reporting Reforms Project: Draft Framework - Principles and Criteria 


• The second is that indicators at the output and outcome level are 
underdeveloped. 


5 Objectives 


A coherent intergovernmental framework for built environment reporting by cities is 
needed to 

• Reduce duplication and fragmentation in the reporting requirements 

• Reduce inefficiencies in the collection of information 

• Ensure that information collected is of strategic value and that reporting and 
monitoring enables devolution, asymmetry and appropriate supervision 

• Ensure that all roles are co-ordinated and complementary 

• Ensure that all roles reflect a meaningful allocation of responsibilities fit for 
the legislative mandate and design of the institution 

• Maximise utilisation of information collected the means of reporting, by all 
stakeholders, including national, provincial and local governments as well as 
citizens and civil society. 

The framework is thus intended to provide a multiple stakeholder consensus on 
how reporting should work, conceptually, based on a set of agreed principles that 
lie at the heart of the framework. 

6 Conceptual Framework for Built Environment 
Reporting 


This section proposes a framework for built environment performance reporting of 
performance. The framework provides a structure within which to ensure 
completeness, sufficiency and interrelatedness of indicators. It focuses on 'what' we 
should be measuring and reporting and is intended to be a framework that is 
completed over time. 

6.1 The results frame 

National Treasury's (2007) 'Framework for Managing Programme Performance 
Information' (FMPPI) introduces performance information as measures relevant to 
the performance of the state. The FMPPI (2007) explains that indicators are 
"...specified to measure performance in relation to inputs, activities, outputs, 
outcomes and impacts. The challenge is to specify indicators that measure things 
that are useful from a management and accountability perspective. This means 
managers need to be selective when defining indicators" (National Treasury, 2007: 
6 ). 

Thus, in arriving at a common understanding of an indicator as a measure that is 
tracked systematically over time to signal progress toward a target (Morra Imas 
and Rist, 2010: 117), it is helpful to unpack the different levels at which 
measurement can be tracked relevant to government intervention. The following 
provides definitions of the concepts that make up an intervention's sequential logic, 
and provides a useful set of ordinal concepts for showing comparable definitions 
from Morra Imas & Rist's 'Road to Results' (2010) text and the FMPPI (2007). 
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Concept 

FMPPI (2007) 

Morra Imas & Rist (2010) 

Inputs 

All the resources that 

contribute to the production 
and delivery of outputs. "What 
we use to do the work," 
including finances, personnel, 
equipment and buildings. 

Resources that go into a project, 
program or policy (funding, staffing, 
equipment, curriculum materials, and 
so forth) 

Activities 

The processes or actions that 
use a range of inputs to 
produce the desired outputs 
and ultimately outcomes. In 
essence, activities describe 
"what we do". 

What we do. Activities can be stated 
with a verb ("market," "provide," 
"facilitate," "deliver"). 

Outputs 

The final products, or goods 
and services produced for 
delivery. Defined as "What we 
produce or deliver". 

What we produce. Outputs are the 
tangible products or services produced 
as a result of the activities. They are 
usually expressed as nouns. They 
typically do not have modifiers. They 
are tangible and can be counted. 

Outcomes 

The medium-term results for 
specific beneficiaries that are 
the consequence of achieving 
specific outputs. Outcomes are 
"what we wish to achieve". 

Why we do it. Outcomes are the 
behavioural changes that result from 
the project outputs (quit smoking). 
Outcomes can be increased, decreased, 
enhanced, improved or maintained. 

Impacts 

The results of achieving 
specific outcomes, such as 
reducing poverty and creating 
jobs. 

Long term changes that result from an 
accumulation of outcomes. Can be 
similar to strategic objectives 


Fronn the above there are clear synergies between existing policy and commonly 
accepted international definitions of the key concepts related to indicators. The 
international definitions do provide more detail, which when read in conjunction 
with the FMPPI are instructive in terms of the task at hand. 

The FMPPI (2007) also provides for "direct" indicators, as well as indicator 
measures that span across the intervention levels mentioned above, including 
economy, efficiency, effectiveness and equity indicators, although these are less 
well defined. Nevertheless, the following figure captures well the various levels at 
which indicator measures can be found which can inform this discussion: 
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Indicators 


Indicators . 



Equity 


Indicators . 
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OUTPUTS 
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Figure 2: Indicator levels across an intervention and their orientation 


6.2 The Results Framework at City Scale 




1 5 


Figure 3: Results Framework at City Scale 
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The diagrann above frannes the results chain for the city administration, describing 
municipal inputs, which through municipal activities and processes are transformed 
into services. These interact on the external system, mainly people and 
communities and produce a layering of outcomes. The ultimate outcome is referred 
to as impact. Both outcomes and impact are not entirely attributable to city 
administration outputs, but are also influenced by externalities from national and 
provincial governments, business and civil society. 

This results chain is unpacked further. 

6.3 Unpacking the enablers 

The following schematic proposes a structural framework for understanding the 
enablers for built environment functions: inputs, activities and processes and 
outputs. 


Inputs Activities & Processes Outputs 



Figure 4: Framework for Built Environment Enablers 

The schematic above suggests that cost is the underlying input beneath that of 
staffing, necessary systems, own infrastructure (buildings and vehicles), 
information, partnerships and bulk resources purchased. These are transformed 
through the following processes into outputs: 

• Infrastructure delivery processes create new infrastructure 

• Asset management processes renew and maintain existing infrastructure 

• Service delivery processes ensure an operating service 

Four sets of meta-activities hold this system together through management, 
financing, planning and governance. The latter two are focused on the broader 
results chain and are not confined to the input-output chain. 

6.4 Unpacking outcomes for the built environment 

Below is a proposed framework for outcomes that reflects the layering of outcomes 
in a hierarchy with regard to built environment services: 
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Figure 5: A Framework for layered outcomes for built environment services 


The following is suggested as a framework for this: 

• Access 

• Utilisation 

• Affordability (Tariff- & Levy-funded) 

• Quality or Continuity of supply (system downtime) 

• Condition of infrastructure 

• Sustainability (Resource Efficiency & Environmental Impact) 

• Customer service 

• Citizen Satisfaction 


6.5 Unpacking Impact 

Impact for built environment services ultimately is about whether citizens' lives are 
improved. Longevity and prosperity are key dimensions of this and can be measure, 
on their own or through indices such as the human development index (HDI). 
However this framework for ultimate impact could also incorporate perceptions on 
quality of life, already being measured by cities, and potentially as some countries 
do, happiness. 


7 Design Principles 


This section intends to propose a set of key design principles that need to inform 
the ongoing development of each system and the 'system' as a whole. It is 
important that these principles are supported and that stakeholders and their 
individual systems align to these principles. 



Reporting Reforms Project: Draft Framework - Principles and Criteria 


7.1 Background to the principles 

In order to identify a set of principles to inform the selection and development of a 
set of indicators and the overall system for the built environment in South Africa's 8 
metropolitan municipalities, it is important to firstly agree on what is meant by an 
indicator. This includes a description of the typologies and varieties, as well as the 
characteristics of good indicators. Much of the international literature in this area is 
borne out of drive to establish and implement results-based monitoring & 
evaluation systems. An identification of these principles therefore cannot be 
complete without first understanding the general overarching concepts, definitions 
and thinking around results-based M&E systems internationally, contextualising it 
locally within South African policy frameworks, and considering the intended 
application for built environment. A common set of principles will therefore be 
distilled to inform the selection and development of indicators for a rationalised, 
streamlined set of reporting arrangements for the built environment. 

7.1.1 Defining an indicator 

Kusek & Rist (2004) define indicators as "... the quantitative or qualitative variables 
that provide a simple and reliable means to measure achievement, to reflect the 
changes connected to an intervention, or to help assess the performance of an 
organization against the stated outcome" (Kusek & Rist, 2004: 65). They can exist 
on multiple levels, in any number of ways, all with different degrees of utility and 
relevance depending on the user. The following quote explains: 

Indicators should be developed for all levels of the results-based M&E 
system, meaning that indicators are needed to monitor progress with 
respect to inputs, activities, outputs, outcomes, and goals. Progress 
needs to be monitored at all levels of the system to provide feedback on 
areas of success and areas in which improvement may be required 
(Kusek & Rist, 2004: 65). 

1.1. 1.1 Principles informing the selection of indicators 

The selection of indicators within this system therefore need to abide by a common 
set principles as well since every piece of data transmitted within the system needs 
to be rationally determined and service a purpose. The indicators, when 
disaggregated, comprise the full set of data which flows through the performance 
monitoring and reporting system and it therefore imperative that they are selected 
in a manner that takes a whole system approach. 

Work by the Leadership Council for Sustainable Development Solutions Network 
(LCSDSN, 2015) is helpful in setting out some broad principles in this regard. The 
following 10 principles have informed the working group on the draft set of 
indicators for the Sustainable Development Goals and the post Millennium 
Development Goals (MDG) agenda: 

1. Limited in number and globally harmonized 

2. Simple, single-variable indicators, with straightforward policy implications 

3. Allow for high frequency monitoring 

4. Consensus based, in line with international standards and system-based 
information 

5. Constructed from well-established data sources 

6. Disaggregated 

7. Universal 

8. Mainly outcome-focused 
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9. Science-based and forward-looking 

10. A proxy for broader issues or conditions (LCSDSN, 2015) 

These ten principles resonate with the same principles set out in the 'Guideline for 
Framing Performance Indicators for the Metros in South Africa' (National Treasury, 
2013). It begins by foregrounding the strategic objectives, or in this case goals, 
and set out the following principles, all compatible with the above: 

• In order to limit the administrative burden, the emphasis should be on fewer 
indicators clearly linked to spatial transformation, rather than many 
indicators that measure performance across a wide range of specific sectors 
of service delivery, housing or infrastructure; 

• The emphasis should be outcome indicators, rather than inputs and outputs 
to give municipalities some flexibility and discretion to adapt their specific 
strategies to local conditions and local priorities, and avoid micro- 
management from above; 

• Each indicator should not require undue original effort to collect the relevant 
data, and it should be feasible to repeat the exercise over time; 

• Indicators should be reasonably simple, understandable and meaningful to 
the municipality; 

• They should be replicable, i.e. independent of the actors using them. They 
should have an unambiguous definition and be capable of objective 
measurement; 

• Indicators should be timely (i.e. record performance within a reasonable 
time-frame) and be capable of showing a trend over time; 

• They should be sensitive to a change in performance, rather than broad- 
brush. 

• Indicators should be 'action-worthy' (World Bank, 2012), i.e. help to answer 
the question "which direction is better?", thereby pointing to how decisions 
might be improved to achieve better urban development outcomes. 

• They should be 'actionable', i.e. specific enough to point towards policy 
actions that can address the problem. 

• The adoption of indicators should be incremental, to build on existing 
frameworks and systems, and allow time for measurement technologies to 
be introduced (National Treasury, 2013). 

Where there is the possibility of tension between high-level, outcome indicator 
focussing on the urban form and built environment with those lower-level indicators 
(National Treasury, 2013: 27), the orientation should be towards the end user, 
namely the people of the metro, with recognition that lower level indicators may 
still be collected but do not make up the core data elements within this built 
environment performance monitoring and reporting system since it is expected to 
operate at a strategic level. 

Thus, when we apply the above at metro level a broad set of principles should 
inform indicator selection overall. These customized principles include: 

1. Limited in number and harmonized across metros 

2. Mainly focused on the four built environment goals and associated results 

3. Constructed from established data sources insofar as possible, with identification 
of new data sources as appropriate 

4. Simple, single-variable indicators, with clear built environment policy implications 
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5. They should be scientifically-based, objectively measured, replicable and forward 
looking 

6. They should allow for high frequency monitoring at regular intervals 

7. They should be disaggregated by locality, allowing for measures of sensitivity in 
performance 

7. Universal applicability for all metros that meet pre-conditions 

8. Consensus-based, in line with local standards, regulations and reporting 
requirements 

9. A proxy for other built environment issues or conditions (As adapted from 
LCSDSN, 2015). 

7.2 Proposed Principles 

The following are proposed principles for an integrated intergovernmental reporting 
system for cities. The word system is used here in a broad conceptual sense and 
does not imply one system. These principles are intended to apply to each 
constituent system as well as the aggregate holistic 'system'. The concept of 
systems referred to here are not concrete IT systems, but refer to a systematic way 
of defining, requiring, collecting, analysing and responding to performance data. IT 
systems may form a part of these. 

7.2.1 Complementary 

There currently is a plurality of systems and monitoring initiatives. This is not a bad 
thing. As they will often fulfil different purposes for different stakeholders, diversity 
in local government monitoring and evaluation should be valued, as opposed to 
putting all the M&E 'eggs in one basket'. However, duplication and fragmentation 
should be avoided through proper co-ordination and partnership by stakeholders in 
the sector to ensure that systems are complementary and that as much integration 
as possible can be realised, without compromising diversity. At the level of each 
indicator, there should be no duplicative reporting of indicators from local to 
national. 

7.2.2 Fit-for-purpose 

Purposes of individual systems will differ. Some are intended for exercising 
accountability, regulation and enforcement. Others are intended to foster learning 
and improvement. Individual systems within the national system need to be fit for 
purpose. Their mechanisms of agreement on indicators, collection of data, rigour in 
treating data, the publication and presentation of performance data and the 
resulting consequences need to be designed for purpose. 

7.2.3 Results-based 

While it will be necessary to measure inputs and activities, on aggregate the 
collective reporting system for cities should be results-based and ensure an 
adequate emphasis on outputs, outcomes & impact. Individual systems, owned by 
different stakeholders are likely to have a specific emphasis in the results chain that 
is appropriate to their purpose. 

While it will still be necessary to measure inputs and activities, there needs to be 
clear line of sight to outputs, outcomes and to some extent impact. 
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Sufficient attention needs to be placed on outcomes in the system. This document 
proposes a framework for built environment indicators against the results chain. 

Particularly for systems that prize accountability as its purpose, they need to 
adequately cater for the appropriate use of accountability in the results chain. An 
organisation's inputs, activities and the production of outputs are always within its 
control, by definition. This does not mean that accountability systems should avoid 
outcomes and impact. These systems still need to be outcome-based ensuring line- 
of-sight, in order to assess the ability to plan and lead. While it is difficult to hold 
individuals, like managers, accountable for outcomes, there are still some 
outcomes, sufficiently attributable, that it will be inappropriate for leaders to be let 
off the hook for. Achievement of outcomes is what leadership is needed for. 

It will be important that the Office of the Auditor-General is brought on board in the 
appropriate treatment of these outcomes. 

7.2.4 Simple but Sufficient 

Simplicity is highly prized. The ability to focus on fewer, simply constructed 
indicators that can be used as a proxy for as many issues, is highly aspired to. 

However, it is important that sets of measures are sufficient to assess performance 
and that the measures selected do not provide a skewed, selective and incomplete 
assessment of performance. 

Where viable proxy indicators that are sufficiently relevant and attributable and 
provide a comprehensive picture of a range of issues, should be explored for 
simplicity. 

7.2.5 Distinguish performance and capacities 

If capacity is the 'means' to achieve goals (UNDP), then what is the relationship 
between capacity and performance (achievement of goals)? It is argued that both 
'means' and 'will' are required in the achievement of goals. Leadership is defined 
here as behaviours that exercise the 'will' to achieve goals, while overcoming 
constraining conditions and circumstances. It is not only confined to leaders. 

This definition of leadership places it in an important position of mediating capacity 
to realise performance as indicated diagrammatically below. It is treated as 'black - 
box' not often seen by the external world, difficult to codify, regulate or measure. It 
is generally apparent whether it is there or not, like when low capacity institutions 
perform well or conversely when high capacity institutions fail. 
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Figure 6: Distinction between performance and capacities 

In the diagram above, human and organisational capacity are separated from that 
of institutional and societal capacity, as the latter set are outside the control of the 
organisation being viewed and form the external context which leaders and the 
organisations can influence, although indirectly and lagging over time. 

It is important to conceptually distinguish city performance from human and 
organisational capacities (inputs) and institutional and societal capacities (context). 

7.2.6 Powers and functions focused 

Performance reporting for cities and local government in general should be guided 
by and focused on their authorised powers and functions. Some flexibility will need 
to be provided for functions that are in the process of devolution. 

It is well known that all municipalities are not alike, and there is a diversity of 
contextual conditions and powers and functions, warranting differentiation e.g. 
Cities in this case. 

However universal systems and approaches are also valued and systems should not 
be unnecessarily differentiated. Indicators and assessments systems should be as 
universal as possible, complemented by differentiation where necessary. Accepting 
extreme differentiation means that it is not possible to compare the performance of 
any two municipalities, as they are different. Many of the expectations of local 
government, irrespective of context are universal, rooted in our bill of rights and 
the rest of the constitution. 
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7.2.7 Triangulation and Balance 

Triangulation refers to richness that a measurement or assessment process can 
have if it relies on utilising multiple sources that are as divergent as possible. 
Assessments that rely on data reported by managers provide a limited perspective. 
A richer assessment is provided when it involves data from politicians, staff, 
citizens, civil society, other spheres of government, academia and regulators. This 
is even further enriched when a variety of data collection methodologies is used 
that covers surveyed perspectives, direct measurement, and observation. In effect 
this principle underpins the point that diversity, of sources and methodologies, 
enrich and make more robust, the overall assessment. 

7.2.8 International Alignment 

Particularly for cities, which are emerging actors globally, there are common global 
expectations. It is important to be able to assess our cities in comparison with other 
cities, particularly with regards to emerging common expectations such as the 
Sustainable Development Goals to replace the Millennium Development Goals. 
International alignment at the same time needs to be complemented by home- 
grown indicators of domestic importance where these are not sufficiently addressed 
in universal global indicators. 

7.2.9 Reciprocation 

Respect, particularly intergovernmental respect is an important principle of these 
systems. Ad-hoc, urgent crisis and poorly designed reporting, exercised through 
hierarchical relationships, is a regular example of disrespect. So too is a lack of co- 
operation with well-conceived and introduced systems. 

Reciprocation refers to the importance of giving back. Those that are reporting 
should find both the reporting process and its outcome valuable. At the very least 
they should receive feedback and preferably analytical and comparative feedback 
that adds value to their planning. 

7.2.10 Open & Transparent 

It is appreciated that there may be circumstances where performance information 
should not be made public, particularly where it concerns performance of 
individuals not in leadership positions. However the default should always be that 
performance information should always be publically shared or available, unless 
there is rational and justifiable reason why it would be unfair or prejudicial to do so. 

7.2.11 Collection at most appropriate scale 

Where similarly specified data needs to be collected at scale larger than cities or 
across all municipalities, this should ideally be done at higher scale on behalf and in 
partnership with local government. Particularly stats SA has a critical role to play of 
partnering with local government in collecting consistent household information for 
the measurement of outcomes of the built environment functions. 

As far as is possible data should be collected in a way that it can be disaggregated 
below city scale. The most important disaggregation is spatially into spatial units 
below that of the city such as regions, integration zones, with wards and EA's being 
the most desired and basic form of spatial disaggregation. However disaggregation 
of data by vulnerable groups is also desired to ensure that achievement of targets 
at city scale is accompanied by similar achievement for vulnerable groups. 
Vulnerable groups here would be specific to the indicator. For example Average trip 
time in peak traffic, could be disaggregated for learners and for disabled people. 
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8 Criteria for a good performance indicator 


The section proposes criteria for consistent selection of good indicators. Selected 
indicators will need to adequately comply with all criteria. 

8.1 Background to the criteria 

Existing policy is also a useful starting point in terms of a common set of principles 
for constructing a good performance indicator. The FMPPI (National Treasury, 2007: 
7) set out that a good performance indicator should be: 

Reliable: the indicator should be accurate enough for its intended use and respond 
to changes in the level of performance. 

Well-defined: the indicator needs to have a clear, unambiguous definition so that 
data will be collected consistently, and be easy to understand and use. 

Verifiable: it must be possible to validate the processes and systems that produce 
the indicator. 

Cost-effective: the usefulness of the indicator must justify the cost of collecting 
the data. 

Appropriate: the indicator must avoid unintended consequences and encourage 
service delivery improvements, and not give managers incentives to carry out 
activities simply to meet a particular target. 

Relevant: the indicator must relate logically and directly to an aspect of the 
institution's mandate, and the realisation of strategic goals and objectives (National 
Treasury, 2007: 7). 

These principles are similar to another set of widely accepted international criteria 
applying the acronym CREAM. The CREAM of selecting a good performance 
indicator is the following recognized criteria (Schiavo-Campo, 1999 in Kusek & Rist, 
2004: 68): 

Clear- Precise and unambiguous 

Relevant- Appropriate to the subject at hand 

Economic- Available at a reasonable cost 

Adequate- Provide a sufficient basis to assess performance 

Monitorable- Amenable to independent validation (Kusek & Rist, 2004: 68). 

This set of criteria is mostly consistent with the FMPPI, really just compounding 
reliability and verifiability into the more ambiguous 'monitorable'; both of which 
share similarities with StatsSA's eight dimensions of data quality in SASQAF (2010). 
StatsSA applies a similar set of principles used in the selection of statistics: 

Relevance- extent to which it meets the needs of clients and sheds light on the 
issues most important to users 
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Accuracy- is the degree to which it describes the phenonnena it was designed to 
measure. It is the closeness between the estimated and true (unknown) values. 
Measured by sampling error and non-sampling error. 

Timeliness- refers to the delay between reference point to which the data pertains 
and the date on which it becomes available, as well as the frequency and 
punctuality. 

Accessibility- refers to the ease with which it can be obtained from the agency, 
including the suitability of the form and medium, including cost considerations. 

Interpretability- refers to the ease with which users can understand statistical 
information through the provision of metadata. 

Coherence- reflects the degree to which it can be successful brought together with 
other statistical information in a broad analytical framework. 

Methodological soundness- refers to the application of international, national or 
peer-agree standards, guidelines and practices for statistical outputs. 

Integrity- refers to the values and related practices that maintain confidence in 
the agency producing statistics and ultimately the statistical product (StatsSA, 

2010: 4). 

The SASQAF data quality dimensions introduce a greater degree of technical rigour, 
unpacked across multiple indicators and levels for each of the above dimensions. 
However, they are useful for benchmarking purposes and help to take stock of the 
principles that all indicators should ultimately endeavour to meet on an individual 
basis. 

Referring back to the FMPPI (2007), attempts to ensure that indicators used in 
government planning meet the originally proposed criteria are best reflected in 
proposed descriptions for performance information included in the 'Framework for 
Strategic Plans and Annual Performance Plans,' Annexure E- Technical indicator 
descriptions and examples (National Treasury, 2010). This definition outline 
assumes that by setting out the following, that the aforementioned principles would 
be observed: 

Table 1: Technical indicator description (National Treasury, 2010: 42} 
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Indicator title 

Identifies tlie title of the strategic outcome oriented goal, objective or programme performance 
indicator 

Short definition 

Provides a bnef explanation of what the indicator is. with enough detail to give a general 
understandmg of the indicator 

Puinose impoft.ince 

Explains what the indicator is intended to show' and whv it is important 

Source collection of data 

Describes where the information comes fi^om and 

how' it is collected 

Method of calculation 

Describes clearly and specifically how' tlie indicator is calculated 

Data limitations 

Identifies any limitation with tlie mdicator data, mcludmg factors that might be beyond the 
department s control 

T\'pe of indicator 

Identifies whether the indicator is measuring inputs, activities, outputs, outcomes or intact, or 
some other dimension of performance such as efficiency, economy or equitv' 

Calcuhition t\‘pe 

Identifies whether the repotted performance is cumulative, or non-cumulatis e 

Reporting cycle 

Identifies if an indicator is reported quarterly, annually or at longer time intervals 

New indicator 

Identifies whether the indicator is new. has significantly changed, or contmues without change 
from the previous year 

Desired performance 

Identifies whether actual performance that is higher or low'er than targeted performance is 
desirable 

Indicator responsibilitr 

Identifies who is responsible for managmg and reporting the mdicator 


The above provides a sound starting point for an individual indicator description, 
but this should of course be amended as appropriate to ensure that the exercise of 
describing the indicator is itself sufficient for meeting the conditions on which an 
individual indicator should be judged. 

8.2 Proposed criteria for good indicators 

Thus, in agreeing on how to select a specific indicator for the purpose of a built 
environment performance monitoring and reporting system, the following distilled 
criteria are proposed: 

• Precise definition - the indicator needs to have a clear, precise definition 
that is unambiguous 

• Verifiable - indicators which are within the direct control of the municipality 
(inputs, activities and outputs) and come from established and trustworthy 
sources are verifiable by nature, yet outcome indicators have a broader 
range of factors which influence them and are not necessarily verifiable 

• Cost-effective - the cost of sourcing and collecting the indicator must be 
commensurate with its utility within the system in relation to other 
indicators 

• Relevant - the indicator must relate logically and directly to the overarching 
goals and results of built environment interventions for people in metros. It 
must be sufficiently, not necessarily entirely, attributable to the entity 
being assessed. 

• Accurate - the indicator title, definition and means of sourcing the data 
must accurately reflect the phenomena it purports to measure. It should be 
as sensitive as possible to change in performance. However insensitive 
indicators such as yes/no compliance indicators are sometimes the most 
sensitive measures available and thus adequate. 

• Timeliness - the indicator must be able to be collected at regular intervals 
without substantial delays interrupting the frequency with which the data is 
available. Timeliness is relative to the purpose that the indicator is being 
measured for. 
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• Appropriate - the indicator must be useful for guiding improvements in 
service delivery and have been formulated in a manner that mitigates the 
risk of unintended consequences or perverse incentives associated with its 
achievement, and complements the existing indicator set in relation to the 
overarching goal. 

• Indicators must be a meaningful reflection of performance. This means 
that there must be a clear relationship between the indicator and 
performance. Improved or declining performance should directly related to 
movements in the indicator measurement. 

• Comparable - Performance should be universally comparable across 
geographical space and overtime. Indicators should be comparable across 
all municipalities or at least within a selected category of municipalities. 
Indicators that are a comparison with a municipality's own target or previous 
performance are statistically not comparable across municipalities. 

Indicators should be comparable across time, thus enabling longitudinal 
trends. Event related indicators are not often comparable across time. 

In this way indicators can take the form of simple indicators, even those that 
represent a binary state of compliance, or be sophisticated indices, composed of 
multiple components aggregated together. They can also be represented by trend- 
lines provided that horizontal data exists to provide a longitudinal view overtime. 

9 Protocols 


There is extensive duplication in two areas in particular, which prompts the creation 
of a reporting protocol. A reporting protocol is a standardised set of indicators 
which are reported on, which are available to a number of different stakeholders. 
Standardised reporting protocols could significantly reduce the amount of reporting 
required, as well as create a standard, agreed upon, set of definitions that cities 
report on. 

The areas in which these are particularly appropriate are in service delivery and job 
creation. There may be a case to create a reporting protocol for housing types and 
access, the Department for Human Settlements will need to pursue this further. 

9.1 Service delivery to dwellings 

There are a number of current indicators intended for tracking both access to 
services and provision of new services to dwellings. These are often required by a 
number of national departments with slightly different definitions. A single protocol 
is proposed for reporting on this below. Of note, dwelling are proposed to be the 
units to be used for accounting for access to services and new provided services. 

The following two protocols are proposed for reporting on access to dwelling-based 
services and provision of new services to dwellings. It is intended to be applied to 
services such as: 

• Water provision 

• Sanitation 

• Electricity provision 

• Refuse collection 

• Access roads 
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The actual Level of Service (LOS) and dwelling type which the reporting will be 
based upon will need to be determined by the appropriate department and StatsSA. 

Table 2: State of access to service by dwellings in settlement type 



Urban 

Informal 

Urban 

Formal 

Rural 

Traditional 

Total 

Number of 
dwellings 
with LOSl 






Number of 
dwellings 
with LOS2 






Number of 
dwellings 
with LOSS 






Number of 
dwellings 
with LOS 4 






Total 







Table 3: Newly provided service by dwellings in settlement type 



Urban 

Informal 

Urban 

Formal 

Rural 

Traditional 

Total 

LOSl - New 
Connections 






LOS2 - New 
Connections 






LOSS - New 
Connections 






LOS4 - New 
Connections 






Total 
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9.2 Job creation 

Sinnilarly nnany governnnent progrannnnes require reporting on job creation. The 
protocol proposed below will enable a standardised set of categories in which the 
job creation programmes will report on. 

Table 4: Job creation reporting protocol 



Number 
of men 

Number 

of 

women 

Number 
of male 
youth 
(18-35) 

Number 

of 

female 

youth 

(18-35) 

Number of 
people with 
disabilities 

Job 

opportunities 
created 
through EPWP 

Unskilled 






Semi- 

skilled 






Skilled 






Job 

opportunities 
created 
through XXX 

Unskilled 






Semi- 

skilled 






Skilled 







Each programme which focusses on job creation will require a separate set of rows. 
It is important to note that this is not necessarily an exhaustive list of all the 
options, but is a proposal for the Department who will be a custodian of this 
protocol to start with. 


10 Technical indicator descriptions 


In order for an indicator to be reported on consistently and accurately, there needs 
to be sufficient information available about the indicator for the entity or person 
who is reporting on the indicator. It is recommended that every indicator which is 
reported on has a technical indicator specification sheet. 

A proposal for a technical indicator specification sheet is presented below. Not all of 
the fields will be filled in for every indicator. 
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Table 5: Proposal for a technical indicator specification sheet 



Technical indicator description sheet 


Indicator 

short 

name 

Indicator 

title 

Category 

If the indicator 
forms part of a 
category within 
the reporting set, 
it is stated here 

Rationale 

Provides the logic behind the inclusion of the 
indicator as a reporting requirement 

Sub-category 

If the indicator 
forms part of a 
sub-category 
within a category, 
it is stated here 


Definition 

Provides a detailed explanation of what the 
indicator is intending to measure 

Code 

If the 

indicator has 
a number or 
code it is 
shown here 
for reference 

purposes 

Unit of 

measurement 

Unit of 

measurement of 
the indicator 

Indicator 

Formula 

The formula used to calculate the indicator 
(should be stated in terms of the data 
elements below) 

Indicator 

origin 

If this indicator is aiigned to other sets of indicators, 
state its origin, particuiariy if part of an internationai 
reporting set 

Frequency 

of 

reporting 

The frequency that the data will need to be 
reported on 

Notes on 
calculation 

Identifies whether the reported performance is an 
average over the time period, at the end of the time 
period or cumuiative and further notes as to how the 
indicator is caicuiated 

Additional 

notes 

Other information which does not fit in 
another field 

-SI 1 

Data 

Element 1 

Data 

element title 

Source 

Likely source of 
the data element 
(Annual October 
Household 
Survey; Housing 
Subsidy System 
registered housing 
applications in the 
Cape Town metro; 
Valuation roll 
2014, etc.) 

Data 

Element 2 

Data element 
title 

Source 

Likely source of 
the data element 
(Annual October 
Household 
Survey; Housing 
Subsidy System 
registered housing 
applications in the 
Cape Town metro; 
Valuation roll 
2014, etc.) 

Frequency 

of 

collection 

The 

frequency 
that the data 
element will 
need to be 
collected in 
order to 
report on the 
indicator 

Units 

The units that the 
data element will 
be collected in 
(for example, 
households, 
kilolitres of water, 
tons, etc.) 

Frequency 

of 

collection 

The 

frequency 
that the data 
element will 
need to be 
collected in 
order to 
report on the 
indicator 

Units 

The units that the 
data element will 
be collected in 
(for example, 
households, 
kilolitres of water, 
tons, etc.) 

Definition 

Provides a detailed explanation of the definition of the 
data element 

Definition 

Provides a detailed explanation of the 
definition of the data element 

Notes 

Any other notes which may be of importance to the 
data element. 

If the data element is collected by another entity, the 
details of the person responsible at the other entity 
should be reported here 

Notes 

Any other notes which may be of importance 
to the data element. 

If the data element is collected by another 
entity, the details of the person responsible at 
the other entity should be reported here 


Data 

Element 3 

Data 

element title 

Source 

Likely source of 
the data element 
(Annual October 
Household 
Survey; Housing 
Subsidy System 
registered housing 

Data 

Element 4 

Data element 
title 

Source 

Likely source of 
the data element 
(Annual October 
Household 
Survey; Housing 
Subsidy System 
registered housing 
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applications In the 
Cape Town metro; 
Valuation roll 
2014, etc.) 




applications in the 
Cape Town metro; 
Valuation roll 
2014, etc.) 

Frequency 

of 

collection 

The 

frequency 
that the data 
element will 
need to be 
collected In 
order to 
report on the 
Indicator 

Units 

The units that the 
data element will 
be collected In 
(for example, 
households, 
kilolitres of water, 
tons, etc.) 

Frequency 

of 

collection 

The 

frequency 
that the data 
element will 
need to be 
collected in 
order to 
report on the 
indicator 

Units 

The units that the 
data element will 
be collected in 
(for example, 
households, 
kilolitres of water, 
tons, etc.) 

Definition 

Provides a detailed explanation of the definition of the 
data element 

Definition 

Provides a detailed explanation of the 
definition of the data element 

Notes 

Any other notes which may be of importance to the 
data element. 

If the data element is collected by another entity, the 
details of the person responsible at the other entity 
should be reported here 

Notes 

Any other notes which may be of importance 
to the data element. 

If the data element is collected by another 
entity, the details of the person responsible at 
the other entity should be reported here 


11 Recommendations on Reporting Platform 


This section puts forward recommendations for institutional arrangements, systems 
and capacity for building a co-ordinated reporting platform. 



Social 

protection 


Education 


General public 
services 


Recreation 
and culture 


Economic 

affairs 


Housing and 
community 
amenities 


Health 


Public order 
and safety 


Environmental 

affairs 
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All city functions can be described in terms of a function view provided above, 
although this is not an exhaustive picture of each function®. 


11.1 Institutional Responsibilities 


Impact 

r ^ 

National Urban 

Integrated Outcomes 

Observatory 

Functional Outcomes 

f \ 

Relevant Transversal or 
Sector Department 

V J 

Functional Outputs 

Functional Activities 

Functional Inputs 

Functional Costs 

Treasury 


11.1.1 Functional Costs and associated concerns 

It is expected that National Treasury will be primarily interested in costs per 
function and associated concerns related to value for money, efficiency and cost- 
effectiveness. As such National Treasury will continue to monitor budgeting and 
expenditure, going forward, against a Standard Chart of Accounts. 

11.1.2 Functional Inputs to Outcomes 

It is assumed that relevant transversal departments e.g. COGTA with reference to 
Governance, Administration and Planning and Treasury with regard to financial 
management as well as each sectoral department will be inclined to closely monitor 
functional inputs to functional outcomes. These departments will still remain the 
custodians and regulators of functional performance information. 

11.1.3 Integrated Outcomes and Impact 

Integrated outcomes and impact measures have traditionally not been measured. 
Over recent years the City Support Programme, the Department of Rural 
Development and Land Reform, and most recently through the lUDF, COGTA has 
indicated an interest in this set of indicators. To some extent this body of indicators 
has also been pursued by the South African Cities Network over the last decade, 
albeit in a non-regulatory capacity, in attempts to collect data for its State of the 
Cities publications. 


^ This classification of services has been taken from the National Treasury's 2011 Budget Review: 
Structure of the Government Accounts 

(http://www.treasurv.Qov.za/documents/national%20budaet/2011/review/Annexure%20W2.pdf) 
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The indicators required to adequately measure integrated outcomes and impact are 
most often outside administrative sets of data and municipalities are ill-equipped to 
measure and report on these regularly. The majority of data in this space revolves 
around people's lives and the functioning of society. Much of this information is best 
measured and researched in specific studies as well as measured through 
engagements with citizens and other stakeholders through surveys and other data 
collection methodologies. This document has earlier made the argument that where 
municipalities are to collect the same information from households, citizens or the 
labour force, consideration should be given to collecting this data at a higher scale, 
to achieve economies of scale and comparability. 

While an argument is being made for a national urban observatory function to be 
carried by either an existing institution or new institution, it is clear that they would 
have to work closely with Statistics South Africa. As outcomes data is used to 
incentivise performance through the grant system. Stats SA will have to approve 
this data in terms of the National Statistical Act. There is thus strong grounds to 
explore how a National Urban Observatory might work together with Stats SA, in 
co-ordinating the collection of integrated outcomes data and acting as a repository 
of such information, not only for national departments, but municipalities 
themselves. 

11.2 Reporting Platforms 

There are two major choices for approaches to co-ordinated reporting, particularly 
on functional reporting. 

11.2.1 Co-ordinated reporting on independent platforms 

In this scenario, departments will continue to determine, regulate, collect data, and 
monitor municipalities independently, albeit in some manner that is better co- 
ordinated. 

11.2.2 Shared platform for functional reporting 

While maintaining the independence of departments to make policy, regulate and 
monitor municipalities in terms of specific requirements, a shared platform can 
bring significant efficiencies, co-ordination and integration of available data. A 
shared platform provides an opportunity to not only link different types of data 
closely such as budget data, input data such as staffing and service delivery data, 
but it is also possible to better manage ease-of-use for municipalities coupled with 
incentives from the grant system for ensuring good quality reporting. The shared 
platform could be established at an agency such as Stats SA, if need be. It need not 
function on a compulsory basis of all departments engaging with municipalities and 
could start with those willing to share a platform. The shared platform will need 
appropriate governance arrangement to ensure it is owned by all the departments 
who share the platform. 

11.3 Co-ordination to be institutionalised 

In order to ensure that reporting reforms are sustained and improved, co- 
ordination of local government reporting should be institutionalised. It is 
recommended that a Council for Local Government Performance Information is 
established. Its role will be to maintain the local government performance 
information system, by upholding the principles and criteria in this document as 
well as progressively realising the conceptual framework. Motivations for new 
reporting asked of local government would need to be approved by this council. 
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which will ensure adherence to the principles, criteria and framework. It could 
function in a similar way to that of the Health Research Advisory Council. Members 
of the council should be nominated by sector departments, SALGA and Cities 
Network. All members should be considered experts in local government reporting 
and indicators and be prepared to attend quarterly meetings. 


12 Conclusion 


In conclusion, the above sets out in broad terms a conceptual framework for 
performance indicators for the built environment, based on a set of principles for 
indicator and reporting systems, criteria for indicators and conceptual framework 
for built environment indicators. It is complemented by proposals for institutional 
arrangements and responsibilities in order to provide an intergovernmental 
framework for built environment reporting amongst cities. 
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